Table of Contents
This chapter provides information on managing clusters.
Clustering forms the basis of the Data ONTAP 10.0 system's performance scaling and global name space. Every node in a Data ONTAP 10.0 system must be part of a cluster. A cluster can contain one or more nodes; however, a node can belong to only one cluster at a time.
Data ONTAP 10.0 clustering is based on the following concepts:
A quorum is formed when a majority of the known nodes, or nodes, in a cluster are healthy and in contact with one another. In this healthy state, they run a voting protocol that elects one node as the master; the remaining nodes are known as secondary nodes. When quorum is formed, it is maintained by continual voting of the nodes. Because of the majority requirement, there can be only one quorum at a time. Other nodes can join an existing quorum (for example, on restart). In a cluster's steady, healthy state, it is normal for a quorum to exist.
When a new quorum forms, a distributed recovery algorithm synchronizes all of the nodes. When this process successfully completes, all of the member nodes go online. When a new node joins an existing quorum, it individually proceeds through the recovery process and goes online.
In a steady state in a well-connected network, healthy nodes are online and each node is capable of running all defined Data ONTAP 10.0 system operations. Changes made to one node are simultaneously propagated to all online nodes. When a node is offline (for instance, if there is no quorum, or if the node is unable to contact the quorum), it is not capable of running all defined Data ONTAP 10.0 system operations.
In a cluster with an even number of nodes, epsilon enables a quorum to form by using slightly weaker requirements. Epsilon is an extra fractional voting weight held by, at most, one node. For example, two out of four eligible nodes are sufficient if one of those two nodes holds epsilon. Epsilon is a configuration setting for a node in a cluster.
An eligible node is one that has been designated by the administrator as an active participant in the cluster. Its presence impacts on the quorum membership requirements, and it is online (if at all possible). An administrator can choose to make the node ineligible for many reasons, such as recent software upgrades, network reconfiguration, and coping with failures.
Managing clusters requires a working familiarity with the concepts that provide the framework for cluster technology in the Data ONTAP 10.0 system. The primary concept that needs to be understood is the maintenance of quorum. Quorum is a precondition for a fully functioning cluster. When quorum is lost, the cluster loses the ability to accomplish normal cluster operations.
For single-node clusters, a majority is achieved when the single node is configured with epsilon. Achieving majority defines whether or not quorum is maintained; without epsilon, the majority is lost for the single-node cluster.
For this reason, a two-node cluster presents some unique challenges in maintaining quorum. In a two-node cluster, epsilon is not held by either node; instead, both nodes are continuously polled to ensure that if one node fails, the other has full read-write access to data as well as access to virtual interfaces and management functions.
In larger clusters, the effect of epsilon is most often felt when the cluster can be partitioned. In general, assuming reliable connectivity among the various nodes of the cluster, a larger cluster is more stable than a smaller cluster. The quorum requirement of a simple majority of half the nodes plus epsilon is easier to maintain in a cluster of 50 nodes than in a cluster of two nodes. However, when the connectivity between two equal portions of a large cluster fails, the group of nodes containing epsilon maintains quorum, assuming that all nodes are healthy. For example, if a single link is established between 25 nodes in one room and 25 nodes in another room to comprise a 50-node cluster and the link fails, then the group of nodes that holds epsilon maintains quorum.
Only one collection of nodes can have quorum at any one time because all of the nodes collectively share a single view of the data. Therefore, if two noncommunicating nodes are permitted to modify the data in divergent ways, it is no longer possible to reconcile the data into a singular data view.
The basis of clustering is the replicated database (RDB). An instance of the RDB is maintained on each node in a cluster. There are a number of processes that use the RDB to ensure consistent data across the cluster; these processes include the management application (mgmt), volume location database (VLDB), virtual-interface manager (vifmgr), and SpinAuth authentication (spinauth). A replication ring is a set of identical processes running on different nodes in the cluster. For instance, the VLDB replication ring for a given cluster consists of all instances of VLDB running in the cluster. RDB replication requires healthy cluster links among all nodes in the cluster; if the cluster network fails in whole or in part, file services can become unavailable. The cluster ring show command displays the status of replication rings and can assist with troubleshooting efforts.